Enhancing information in a three-dimensional map

ABSTRACT

Implementations generally relate to enhancing information in a three-dimensional map (3D). In some implementations, a method includes determining, using a client device, user information associated with a user. The method further includes generating a 3D map of a physical environment based on the user information. The method further includes determining relevant information based on the user information. The method further includes displaying the relevant information in the 3D map in a user interface of the client device.

BACKGROUND

Three-dimensional (3D) maps provide a user with a representation of aphysical environment. 3D maps represent internal environments (e.g.,indoors) or external environments (e.g., outdoors). 3D maps may begenerated using specialized optical display devices. In some cases, suchdevices are mounted on or around the user's head. In other cases, suchdevices may be hand-held devices such as smartphones. 3D maps havevarious applications. For example, a 3D map may be used in video gamesystems, augmented reality systems, etc.

SUMMARY

Implementations generally relate to enhancing information in athree-dimensional (3D) map. In some implementations, a method includesdetermining, using a client device, user information associated with auser. The method further includes generating a 3D map of a physicalenvironment based on the user information. The method further includesdetermining relevant information based on the user information. Themethod further includes displaying the relevant information in the 3Dmap in a user interface of the client device.

A further understanding of the nature and the advantages of particularimplementations disclosed herein may be realized by reference of theremaining portions of the specification and the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an example physical environment,which may be used for some implementations described herein.

FIG. 2 illustrates a diagram of an example user device being worn by auser, according to some implementations.

FIG. 3 illustrates an example flow diagram for enhancing information ina 3D map, according to some implementations.

FIG. 4 illustrates a block diagram of an example augmented realityenvironment, according to some implementations.

FIG. 5 illustrates a block diagram of an example augmented realityenvironment, according to some implementations.

FIG. 6 illustrates a block diagram of an augmented reality system,according to some implementations.

FIG. 7 illustrates a block diagram of an example network environment,which may be used for some implementations described herein.

FIG. 8 illustrates a block diagram of an example computing system, whichmay be used for some implementations described herein.

DETAILED DESCRIPTION

Implementations described herein build a 3D map using an augmentedreality (AR) user device such as AR glasses. As described in more detailherein, implementations provide an AR user device that has a camera ordevice that measures 3D depth in the real world, physical environment.

In some implementations, a system determines user information associatedwith a user, using a client device such as an AR user device. The systemfurther generates a 3D map of a physical environment based on the userinformation. The system further determines relevant information based onthe user information. The system further displays the relevantinformation in the 3D map in a user interface of the client device.

FIG. 1 illustrates a block diagram of an example physical environment100, which may be used for some implementations described herein. Shownare a building 102 and a user 104. In this particular example scenario,user 104 is looking up at building 102. Various example implementationsinvolving user 104 in physical environment 100 are described in moredetail herein.

For ease of illustration, FIG. 1 shows a simplified example of physicalenvironment 100, with a simplified block representing building 102. Thisblock may represent any number of buildings and/or objects in physicalenvironment 100. In other implementations, physical environment 100 mayhave other objects including other types of objects instead of, or inaddition to, those shown herein.

FIG. 2 illustrates a diagram of an example user device 202 being worn bya user 104, which may be used for some implementations described herein.In various implementations, user device 202 includes display screens 220and cameras 222. In some implementations, display screens 220 may useorganic light-emitting diode (OLED) technology, including waveguideoptics and/or bulk optical elements to focus and guide light fromorganic LEDs. In other implementations, other types of light sources andoptics may be used. In various implementations, display screens 220enable user 104 to view the actual, physical environment, and alsoenable the user to view virtual images superimposed on the viewedphysical environment.

In various implementations, user device 202 may also include varioussensors that facilitate in determining movements and activities of user104. Various implementations directed at determining movements andactivities of user 104 are described in more detail herein. In someimplementations, user device 202 may include a head position sensor 224and a head orientation sensor 226, which are shown integrated into userdevice 202. In some implementations, one or both of head position sensor224 and head orientation sensor 226 may be separately mounted on otheruser devices worn by user 104. In other implementations, user device 202may not have all of the components shown and/or may have othercomponents including other types of components instead of, or inaddition to, those shown herein. In other example implementations, userdevice 202 may include a 3D depth camera, an eye tracking device, etc.In some implementations, user device 202 may communicate with auxiliarydevices such as pedometers, activity trackers, etc.

In various implementations, user device 202 is a part of an AR system,which displays one or more virtual images in display screens 220viewable by user 104. In various implementations, the AR system mayachieve this by controlling pixels of display screens 220, through whichuser 104 observes the external environment. As described in more detailherein, user device 202 enables user 104 to view the actual physicalenvironment while simultaneously viewing virtual images that aresuperimposed on the view of the physical environment.

In some implementations, user device 202 may receive voice commands andrequests from user 104. In some implementations, user device 202 mayreceive commands and requests in the form of hand gestures from user104. In some implementations, user device 202 may act on behalf of user104 without user input. For example, user device 202 may automaticallyprovide information to user 104 (e.g., alerts, recommendations, etc.).

In various implementations, user device 202 is a wearable device. Asshown, for example, user device 202 may be worn on the head of user 104(e.g., as glasses). While some implementations are described herein inthe context of user device 202, which is a head mounted device, theseimplementations and others may also apply to a variety of differenttypes of user devices (e.g., headset, watch, smart phone, otherelectronics, etc.). Various user devices are possible, and theparticular type of user device will depend on the particularimplementation.

FIG. 3 illustrates an example flow diagram for enhancing information ina 3D map, according to some implementations. Referring to both FIGS. 1,2, and 3, a method is initiated at block 302, where the system such as aclient device (e.g., user device 202 of FIG. 2) determines userinformation associated with a user. In various implementations, the userinformation includes information unique to user 104. As described inmore detail herein, the system provides relevant information to the userbased on the user information.

In some implementations, the user information may include 3D depth data,where the 3D depth data is collected by the client device (e.g., userdevice 202 of FIG. 2). In various implementations, the 3D depth data isbased on the position of user 104 in the physical environment and fromthe point of view of user 104. For example, referring to FIGS. 1, 2, and3, a user camera may include a 3D depth camera that determines the depthof objects in front of user 104, and more specifically, in the field ofview of the depth camera of user device 202. Any suitable depth cameramay be used.

In some implementations, the user information may also include userattention information. In some implementations, the system determinesthe user attention information based on user gaze information. Forexample, in some implementations, user device 202 of FIG. 2 may includean eye tracking device that determines eye positions and/or eye movementbased on measurements of point of gaze of the user (e.g., line ofsight). Any suitable eye tracking device and/or head orientationtracking device may be used.

In some implementations, the user information may also include useractivity information. In some implementations, the activity informationmay include categories of activity. For example, categories may includestationary activities (e.g., being home, being at work, etc.).Categories may also include movement activities (e.g., walking, running,biking, driving, etc.).

In some implementations, the activity information may also includepredetermined activities (e.g., walking, running, biking, driving,etc.). Any given predetermined activity may fall into a particularcategory of activity.

In some implementations, the user information may also include locationinformation or geographical information. In some implementations, userdevice 202 may collect sensor information to determine the location andactivity of user 104. Such sensor information may include data from aglobal positioning system (GPS), accelerometer, compass, etc.

In some implementations, the system may include user input. For example,in some implementations, the system may utilize any suitable handgesture recognition techniques to recognize hand gestures. Theparticular hand gestures may vary and will depend on the particularimplementation. For example, in some implementations, the user making aparticular gesture with respect to a particular AR object or icon mayhave a predetermined meaning. For example, the user touching aparticular AR object or icon may mean, “select.” In another example, theuser flicking AR icon may mean, “cancel.” Using such hand gesturesand/or other gestures may add more information to the 3D map via AR. Forexample, if a large (predetermined) number of users touch the same ARicon in the same area or position in the 3D map, the system maydetermine that such AR information is meaningful or relevant. In thiscase, the system may automatically display such relevant AR informationand/or send such meaningful data to another device or service (e.g.,smartphone, self-driving car, 2D map, etc.).

At block 304, the system generates a 3D map of the physical environmentbased on the user information. In some implementations, a 3D map may bebased at least in part on the 3D depth data of the user information. Insome implementations, the 3D map may include environmental elements thatuser device 202 detects (e.g., buildings, streets, sidewalks, alleys,etc.). As such, the system generates the 3D map as user 104 moves (e.g.,walks, etc.) around a physical environment (e.g., around a city, etc.).As described in more detail herein, the system provides relevantinformation to user 104 based on the location of user 104 and where theattention of user 104 is place in the physical environment.

In some implementations, the AR system may receive user informationassociated with multiple users, where multiple user devices of differentusers upload user information to the AR system. The AR system mayaggregate and synthesize user information from multiple users in orderto include more data for the generating of the 3D map.

In some implementations, the system may also use two-dimensional (2D)map data to facilitate in generating the 3D map. For example, the systemmay generate a 2D map based on user movement detected by GPS. The systemmay access 2D mapping information from a supplemental source (e.g., amapping service).

At block 306, the system determines relevant information based on theuser information. In some implementations, the determining of therelevant information may be based on one or more relevance policies. Invarious implementations, the system analyzes the user informationprovided by user device 202, and may also include and analyze other datacollected by various sensing devices. As indicated herein, userinformation may include attention information (e.g., based on directionof head, gaze, etc.). User information may also include user activityinformation such as whether the user is walking, running, biking, in acar, riding a bus, etc. User information may also include locationinformation or geographical information (e.g., based on GPS position ofthe user, etc.). User information may also include user-providedcommands based on voice, hand gestures, etc.

In some implementations, a relevance policy may include determining thatinformation is relevant based on time of day and location of user 104.For example, if the time is 11:45 am (e.g., around lunch time) and thelocation of user 104 being in proximity to a building with restaurants,the system may determine a particular advertisement is relevant. Assuch, in some implementations, the relevant information may includeadvertisements. Because such advertisements are relevant to the userinformation (e.g., activity of the user), the advertisements are morelikely to be useful to or desired by user 104.

In another example implementations, a relevance policy may includedetermining that information is relevant based on the current activityof user 104 and the movement of surrounding objects. For example, ifuser is walking toward a blind intersection and a fast moving objectsuch as a car is approaching the intersection, the system may determinethat an alert or warning to the user is relevant. Such an alert mayinclude navigation information (e.g., stop walking or change directionin order to avoid danger). As such, in some implementations, therelevant information may include notifications. The relevant informationmay also include navigation information.

As described in more detail herein, in various implementations, thesystem aggregates such data in order to determine where the user'sattention is in real, physical world and to provide user 104 withrelevant information that may be useful to user 104.

At block 308, the system displays the relevant information in the 3D mapin a user interface of the client device. In various implementations,the system uses map information (e.g., 2D and/or 3D map data) and userinformation in order to determine what information to display to user104. As indicated herein, user device 202 enables user 104 to view theactual physical environment while simultaneously viewing virtual imagesthat are superimposed on the view of the physical environment. Thevirtual image may include objects and/or text that convey information touser 104. For example, a virtual image that is a virtual object mayinclude a logo or other image in an advertisement or an alert orwarning. In some implementations, the virtual image may include textsuch as text in an advertisement or in an alert or warning.

In various implementations, the system, more specifically user device202, presents information in order to inform the user of options. Asdescribed in more detail herein, such options may include purchasingbehavior based on advertisements. For example, if shortly after anadvertisement is displayed user 104 goes to store or restaurantassociated with the advertisement, the system may determine that user104 read the advertisement and acted accordingly.

FIG. 4 illustrates a block diagram of an example augmented realityenvironment, according to some implementations. Shown is building 102and user 104. In this particular example scenario, user 104 is lookingup at building 102, similarly to the scenario described in connectionwith FIG. 1.

As shown, an advertisement 402 is displayed against building 102. Invarious implementations, user 104 views the physical environment throughdisplay screens 220 of user device 202. User 104 also viewsadvertisement 402 virtually on building 102 as if advertisement 402 isactually on building 102 in real life.

In some implementations, the system may determine a direction that user104 may be looking based on the head orientation of user 104, and maydetermine optimal locations for placement of advertisement 402 in theenvironment based on the direction that user 104 is looking. In someimplementations, the system may determine that areas in the scene arecrowded, and may place ads in areas that are not crowded.

In various implementations, the placement of a virtual image such asadvertisement 402 on display screens 220 is based on the attention ofuser 104. For example, user device 202 may display advertisement 402 atthe area where user 104 is currently gazing or where user 104 is oftengazing within a predetermined time period (e.g., 15 seconds, etc.). Thisincreases the chances that user 104 will look at advertisement 402. Insome implementations, user device 202 may move advertisement 402 tomultiple locations, or show advertisement 402 in multiple locations, inorder to bring advertisement 402 to the attention of user 104.

In some implementations, user device 202 may determine whether user 104reads advertisement 402 or not. Such a determination may be based onuser gaze or user behavior (e.g., entry into the advertised store orrestaurant, etc.). In some implementations, user device 202 or the ARsystem may send such information to a source of the advertisement (e.g.,the advertising client).

For ease of illustration, FIG. 1 shows a simplified example of physicalenvironment 100, with a simplified block representing building 102. Thisblock may represent any number of buildings and/or objects in physicalenvironment 100. In other implementations, physical environment 100 mayhave other objects including other types of objects instead of, or inaddition to, those shown herein.

FIG. 5 illustrates a block diagram of an example augmented realityenvironment 500, according to some implementations. Shown are buildings502, a street 504 on which user 104 is walking, and a street 506 onwhich a car 508 is moving.

In some implementations, the system may determine navigation informationbased on user location information, user gaze information, etc. In someimplementations, the navigation information may be used to direct user104 where to go if user 104 is driving a car.

Various implementations described herein may also be used in aself-driving car navigation system. In some implementations, the systemmay collect information associated with the car, where such informationis similar to user information collected for a user. For example, suchinformation may include location information as described herein (e.g.,GPS information, etc.). In various implementations, the system mayaggregate information associated with other cars and user informationassociated with users. In some implementations, the AR system mayreceive information associated with multiple cars, includingself-driving cars, where multiple devices in the different cars uploadinformation to the AR system. The AR system may aggregate and synthesizethe information from multiple car in order to include more data forvarious purposes. For example, such information may be used to monitorthe surroundings (e.g., moving objects) in proximity to the self-drivingcar. The system may use such information to prevent accidents. Forexample, the system may cause the navigation system of the self-drivingcar to be ready to slow down, stop, or turn if another object crossesthe path of the self-driving car. Such information may also be used togenerate the 3D map described herein.

In some implementation, if user 104 is in the self-driving car and user104 is wearing user device 202, the system may utilize information fromboth the information associated with the self-driving car and the userinformation associated with user 104. In various implementations, thelocation information for both the self-driving car and the user 104match, as both are moving together. In some implementations, the userinformation associated with user 104 may include attention information.Such attention information might not affect the self-driving cardirectly but may affect relevant information provided to user 104. Forexample, the system may provide an advertisement to user 104. The systemmay enable user 104 to give commands to the self-driving car navigationsystem if user 104 wants to take action based on the advertisement. Forexample, user 104 may instruct the self-driving car to go to aparticular restaurant in the advertisement.

Referring to FIG. 5, if user 104 is walking along street 504, and car508 is moving on street 506, and both are moving toward the sameintersection, user device 202 may display an alert to user 104 in orderto direct user 104 to stop or to turn in order not to step into the pathof car 508. Similarly, a user device associated with car 508 may directcar 508 to slow down or be ready to stop in case user 104 steps into thepath of car 508.

For ease of illustration, FIG. 5 shows four buildings 502, one user 104,and one car 508. These objects may represent multiple objects of eachtype. In other implementations, augmented reality environment 500 maynot have all of the objects shown and/or may have other objectsincluding other types of objects instead of, or in addition to, thoseshown herein. For example, augmented reality environment 500 may includemultiple cars moving in different directions. Augmented realityenvironment 500 may include multiple people walking or biking.

Although the steps, operations, or computations may be presented in aspecific order, the order may be changed in particular implementations.Other orderings of the steps are possible, depending on the particularimplementation. In some particular implementations, multiple steps shownas sequential in this specification may be performed at the same time.Also, some implementations may not have all of the steps shown and/ormay have other steps instead of, or in addition to, those shown herein.

Embodiments described herein provide various benefits. In particular,embodiments provide a user with behavioral options such as where to gohave a meal, make purchase, etc. Implementations may also provide safetyand/or navigation choices for a user. These benefits may also bevaluable in engaging and maintaining user involvement in any of avariety of activities associated with personal development, health,educational, gaming, and various commercial or business applications.

FIG. 6 is a schematic block diagram of an agent system 600. System 600includes sensor unit 602. In various implementations, agent system 600may reside on and/or be integrated with a user device such as userdevice 202 of FIG. 2. In some implementations, sensor unit 602 mayinclude a microphone 604, a positioning sensor 606, a gyro sensor 608,an accelerometer 610, a magnetic sensor 612, a biosensor 614, and acamera 616.

System 600 also includes a control unit 618. In various implementations,control unit 618 may include an information transformation system 620and an output control system 622. System 600 also includes a memorysystem 624, a speaker 626, a network interface system 628, an inputinterface system 630, and a display 632. Some of these components may beoptional. For example, system 600 may or may not include an integrateddisplay, and system 600 may output information to a display 632 in someimplementations.

For ease of illustration, FIG. 6 shows one block for each type ofcomponent shown. These blocks may represent multiple components of eachtype. For example, there may be multiple speakers. In variousimplementations, system 600 may not have all of the components shownand/or may have other elements including other types of componentsinstead of, or in addition to, those shown herein.

In some implementations positioning sensor 606 may depend on a globalpositioning system (GPS), WiFi, Bluetooth, or other suitabletechnologies. As shown, control unit 618 is operably connected to sensorunit 602, memory system 624, speaker 626, network interface system 628,input interface system 630, and display 632. Control unit 618 alsoincludes one or more processors (not shown) that operates in conjunctionwith instructions and data from memory system 624, network interfacesystem 628, and/or input interface system 630 (simple hardware devicessuch as a power button, a brightness control knob, etc.) to transforminformation gathered from sensor unit 602 into data usable by outputcontrol system of unit 622. In some implementations, this data and otherinformation procured by agent system 600 may be provided to the user viacomponents such as speaker 626, and some data and/or information may beprovided to remote systems via network interface system 628, accordingto the particular applications being run in conjunction with agentsystem 600.

FIG. 7 illustrates a block diagram of an example network environment700, which may be used for some implementations described herein.Implementations described herein may be implemented by a client 702 suchas user device 202 of FIG. 2, or may be implemented by client 702 incombination with a system 704 (e.g., an AR system). In someimplementations, client 702 communicates with system 704. System 704 mayinclude a server device 706 and a database 708. Also shown is a network710 through which system 704 and/or client 702 may communicate withclients 712, 714, 716, and 718 (e.g., other user devices).

For ease of illustration, FIG. 7 shows one block for each variouscomponents of network environment 700. These blocks may representcomponents of each type of component. In other implementations, networkenvironment 700 may not have all of the components shown and/or may haveother elements including other types of elements instead of, or inaddition to, those shown herein.

In the various implementations described herein, client 702 causes theelements described herein (e.g., advertisements, alerts, navigationinformation, and other relevant information) to be provided to the user(e.g., displayed in a user interface on one or more display screens,etc.).

FIG. 8 illustrates a block diagram of an example computing system 800,which may be used for some implementations described herein. Forexample, computing system 800 may be used to implement user device 202of FIG. 2, as well as to perform implementations described herein. Insome implementations, computing system 800 may include a processor 802,an operating system 804, a memory 806, and an input/output (I/O)interface 808. In various implementations, processor 802 may be used toimplement various functions and features described herein, as well as toperform the method implementations described herein. While processor 802is described as performing implementations described herein, anysuitable component or combination of components of computing system 800or any suitable processor or processors associated with computing system800 or any suitable system may perform the steps described.Implementations described herein may be carried out on a user device, ona server, or a combination of both.

Computing system 800 also includes a software application 810, which maybe stored on memory 806 or on any other suitable storage location orcomputer-readable medium. Software application 810 provides instructionsthat enable processor 802 to perform the implementations describedherein and other functions. Software application may also include anengine such as a network engine for performing various functionsassociated with one or more networks and network communications. Thecomponents of computing system 800 may be implemented by one or moreprocessors or any combination of hardware devices, as well as anycombination of hardware, software, firmware, etc.

For ease of illustration, FIG. 8 shows one block for each of processor802, operating system 804, memory 806, I/O interface 808, and softwareapplication 810. These blocks 802, 804, 806, 808, and 810 may representmultiple processors, operating systems, memories, I/O interfaces, andsoftware applications. In various implementations, computing system 800may not have all of the components shown and/or may have other elementsincluding other types of components instead of, or in addition to, thoseshown herein.

Although the description has been described with respect to particularembodiments thereof, these particular embodiments are merelyillustrative, and not restrictive. Concepts illustrated in the examplesmay be applied to other examples and implementations.

In various implementations, software is encoded in one or morenon-transitory computer-readable media for execution by one or moreprocessors. The software when executed by one or more processors isoperable to perform the implementations described herein and otherfunctions.

Any suitable programming language can be used to implement the routinesof particular embodiments including C, C++, Java, assembly language,etc. Different programming techniques can be employed such as proceduralor object oriented. The routines can execute on a single processingdevice or multiple processors. Although the steps, operations, orcomputations may be presented in a specific order, this order may bechanged in different particular embodiments. In some particularembodiments, multiple steps shown as sequential in this specificationcan be performed at the same time.

Particular embodiments may be implemented in a non-transitorycomputer-readable storage medium (also referred to as a machine-readablestorage medium) for use by or in connection with the instructionexecution system, apparatus, or device. Particular embodiments can beimplemented in the form of control logic in software or hardware or acombination of both. The control logic when executed by one or moreprocessors is operable to perform the implementations described hereinand other functions. For example, a tangible medium such as a hardwarestorage device can be used to store the control logic, which can includeexecutable instructions.

Particular embodiments may be implemented by using a programmablegeneral purpose digital computer, and/or by using application specificintegrated circuits, programmable logic devices, field programmable gatearrays, optical, chemical, biological, quantum or nanoengineeredsystems, components and mechanisms. In general, the functions ofparticular embodiments can be achieved by any means as is known in theart. Distributed, networked systems, components, and/or circuits can beused. Communication, or transfer, of data may be wired, wireless, or byany other means.

A “processor” may include any suitable hardware and/or software system,mechanism, or component that processes data, signals or otherinformation. A processor may include a system with a general-purposecentral processing unit, multiple processing units, dedicated circuitryfor achieving functionality, or other systems. Processing need not belimited to a geographic location, or have temporal limitations. Forexample, a processor may perform its functions in “real-time,”“offline,” in a “batch mode,” etc. Portions of processing may beperformed at different times and at different locations, by different(or the same) processing systems. A computer may be any processor incommunication with a memory. The memory may be any suitable datastorage, memory and/or non-transitory computer-readable storage medium,including electronic storage devices such as random-access memory (RAM),read-only memory (ROM), magnetic storage device (hard disk drive or thelike), flash, optical storage device (CD, DVD or the like), magnetic oroptical disk, or other tangible media suitable for storing instructions(e.g., program or software instructions) for execution by the processor.For example, a tangible medium such as a hardware storage device can beused to store the control logic, which can include executableinstructions. The instructions can also be contained in, and providedas, an electronic signal, for example in the form of software as aservice (SaaS) delivered from a server (e.g., a distributed systemand/or a cloud computing system).

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application. It isalso within the spirit and scope to implement a program or code that canbe stored in a machine-readable medium to permit a computer to performany of the methods described above.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

Thus, while particular embodiments have been described herein, latitudesof modification, various changes, and substitutions are intended in theforegoing disclosures, and it will be appreciated that in some instancessome features of particular embodiments will be employed without acorresponding use of other features without departing from the scope andspirit as set forth. Therefore, many modifications may be made to adapta particular situation or material to the essential scope and spirit.

What is claimed is:
 1. A computer-implemented method comprising:determining, using a client device, user information associated with auser; generating a three-dimensional (3D) map of a physical environmentbased on the user information; determining relevant information based onthe user information; and displaying the relevant information in the 3Dmap in a user interface of the client device.
 2. The method of claim 1,wherein the user information includes 3D depth data, wherein the 3Ddepth data is collected by the client device.
 3. The method of claim 1,wherein the user information also includes user attention information.4. The method of claim 1, wherein the user information also includesuser activity information.
 5. The method of claim 1, wherein thedetermining of the relevant information is further based on one or morerelevance policies.
 6. The method of claim 1, wherein the relevantinformation comprises navigation information.
 7. The method of claim 1,wherein the relevant information comprises advertisements.
 8. Anon-transitory computer-readable storage medium carrying programinstructions thereon, the instructions when executed by one or moreprocessors cause the one or more processors to perform operationscomprising: determining, using a client device, user informationassociated with a user; generating a three-dimensional (3D) map of aphysical environment based on the user information; determining relevantinformation based on the user information; and displaying the relevantinformation in the 3D map in a user interface of the client device. 9.The computer-readable storage medium of claim 8, wherein the userinformation includes 3D depth data, wherein the 3D depth data iscollected by the client device.
 10. The computer-readable storage mediumof claim 8, wherein the user information also includes user attentioninformation.
 11. The computer-readable storage medium of claim 8,wherein the user information also includes user activity information.12. The computer-readable storage medium of claim 8, wherein thedetermining of the relevant information is further based on one or morerelevance policies.
 13. The computer-readable storage medium of claim 8,wherein the relevant information comprises navigation information. 14.The computer-readable storage medium of claim 8, wherein the relevantinformation comprises advertisements.
 15. A system comprising: one ormore processors; and logic encoded in one or more non-transitorycomputer-readable storage media for execution by the one or moreprocessors and when executed operable to perform operations comprising:determining, using a client device, user information associated with auser; generating a three-dimensional (3D) map of a physical environmentbased on the user information; determining relevant information based onthe user information; and displaying the relevant information in the 3Dmap in a user interface of the client device.
 16. The system of claim15, wherein the user information includes 3D depth data, wherein the 3Ddepth data is collected by the client device.
 17. The system of claim15, wherein the user information also includes user attentioninformation.
 18. The system of claim 15, wherein the user informationalso includes user activity information.
 19. The system of claim 15,wherein the determining of the relevant information is further based onone or more relevance policies.
 20. The system of claim 15, wherein therelevant information comprises navigation information.